Another approach for actors would be to simply embed them directly in the HAMT (as we currently do).

Adding a tag prefix to actors only makes sense if they are made into separate ipld objects.

Thoughts welcome here too.

I think actors as their own objects makes more sense in the long run

Ideally, we shouldn't make requirements on block boundaries in IPLD dags. That is, users should be able to embed or reference by CID.

Unfortunately, we do this all the time anyways and this requirement is hard-coded everywhere...

@Stebalien yeah, ideally it wouldnt matter, but we live in the world where it unfortunately does :/

99% of the time I’m in the “no requirements on block boundaries” camp, but in the case of Filecoin it would concern me a little bit. allowing for a variations on block boundaries creates another vector to consider in performance and possibly even security. being rigid about the block format seems appropriate.

missing uint8 for completness

what about changes over time? would that result in a new tag? or should there be a version field?

new tag

Is that really worth the few bytes? The object's already going to be at least 55 bytes, likely much more. We can turn this into a map of numeric tags to fields with 5 additional bytes (a single byte key per field).

Really, it's not a huge deal but something to consider.

The 5 bytes arent really worth it for the block, but they are for the messages, and I wanted to have a consistent format.

messages (with IDs) using this format will likely be under 20 bytes, and messages will take up the bulk of data being moved around. adding another 5 bytes is rather painful.

Also, does a map of numeric tags to fields really buy us that much? it ends up looking like a hacky array. (though i guess it does allow us to leave fields out)

Under 20 bytes? Are you somehow skipping fields?

Also, does a map of numeric tags to fields really buy us that much? it ends up looking like a hacky array. (though i guess it does allow us to leave fields out)

Really, this just allow us to write /ipld/... paths. However, we can also just introduce a new pathing scheme, /filecoin/ for filecoin paths. However, this does mean that graphsync will need to support filecoin paths out of the box.

@Stebalien

To and From fields both around 4-5 bytes, nonce will be 1 byte most of the time, value will be a few bytes, up to 8 bytes in some bad cases, but hopefully around 2-3 bytes if we can help it. Method is one byte, and then params will be more. But a simple 'send money' transaction won't have any params.

@Stebalien also, with our own custom CID type, we can translate nice pathing schemes out of it. this is how we do pathing through other blockchain formats (and unixfs :/ )

I think actors as their own objects makes more sense in the long run

Is that really worth the few bytes? The object's already going to be at least 55 bytes, likely much more. We can turn this into a map of numeric tags to fields with 5 additional bytes (a single byte key per field).

Really, it's not a huge deal but something to consider.

Ideally, we shouldn't make requirements on block boundaries in IPLD dags. That is, users should be able to embed or reference by CID.

Unfortunately, we do this all the time anyways and this requirement is hard-coded everywhere...

Any chance we could use multicodecs? Too big?

we could use multicodecs, but i was just trying to fit into the cbor tags table. Unclear if thats worth doing...

I’ll +1 using multicodecs with the caveat that I’m not sure offhand what the difference in size would be.

@mikeal Yeah, Using multicodecs would be nice. Though it moves us out of the realm of using cbor, and into using our own custom encoding format.

given that we already use cbor I would really like to stick with cbor tags, as this ensures cbor implementations are compatible out of the box and this is well defined cbor making other tools work well

i'm a little confused by the array notation in here, does it literally mean an major type 4 array to hold all the data items or does it denote just the sequence of data items? the difference is the presence or absence of major type 4 data item following the outer object tag, which contains the elements of the array.

Yeah, major type 4 to hold all the data items, that way its a valid cbor object.

lets have a very specific name for the left hand side that we use nowhere else so we can refer to things of that type unambiguously. is it object?

how about 'FCE type' for Filecoin Compact Encoding type?

sure, anything that is unique like that works

• Name this something

i think we need to say that encodings must follow the rules for canonicalization per https://tools.ietf.org/html/rfc7049#section-3.9 and that decoders should use strict mode https://tools.ietf.org/html/rfc7049#section-3.10. also decoders should disable unused features like streaming for security's sake, and set explicit limits (suggestions?) on the size of byte arrays, lists etc as a precaution against dos.

absolutely

i would set limits on sizes of objects, canonicalize, strict decoding, and so on. i'll make a mention of that in the doc

• TODO: this

is it a goal to restrict the cbor data item types, eg to not encode to maps? if so we should say so.

what do you mean?

is it the case that we will ever encode something to major type 5? if not and there is a design reason for this eg concern about suitability of key types across languages then we should say so. also if we are not going to use type 5 they can turn it off for safety.

I think that for the filecoin compact encoding format, we can leave that out. Though for the general cbor ipld stuff, people will still want to use that (i.e. in actor storage).

For the purpose of this doc, i think we could explicitly say its not used and should invalidate any object

think we should leave this out until we have use for it

fair

should note that we cannot distinguish between unset/missing and zero values given this encoding (which is fine). the encoder must always encode every field is for every object. maybe that is obvious but doesnt hurt to be explicit.

Yeah, probably worth pointing out. I did like that this scheme removed that ambiguity.

this encoding scheme doesn't allow for forward compatibility so for example if we have a new object version old binaries are not going to be able to read it. this as opposed a scheme where fields are tagged an old implementations can just ignore new ones it doesnt understand. this is a feature i'm guessing, for size efficiency?

this does allow for forward compatibility. Every object is tagged, if you change an object, you pick a new tag.

In general in Filecoin, old binaries will not be able to understand new protocol formats, and it won't make sense for them to try to do so.

though maybe i can see a case where a thing wants to try and read messages, and if we were to add a new field for some weird reason, it wouldnt be able to read the old fields. I don't consider than much of an issue though, as my main concern is that the old impl knows the object is a different type, even if it can't understand it.

how does this encoding scheme look for multiple versions in the code? do you get a types.BlockV1 or types.BlockV2 depending on what you decoded and then have ifs? do you get a types.Block that has fields that might or might not have what you expect and you if on the version which is surfaced maybe in an unserialized field?

practically speaking, a change in block format will happen at a fixed height, and all blocks created after that height will be the new format. disambiguating should be straightforward.

I'm not 100% sure how implementations will handle this, but i would expect a single block type with any new fields added, and a version field the code can check (or just a height check, since it will need to be aware of that anyways)

implementers will need a bunch of examples to ensure their implementation is working. uh, including us i suppose. also prolly a good idea to link to something that makes it easy to decode cbor to diagnostic output.

I'll link this somewhere too, but this tool is really great for debug printing cbor, or even converting it to json

• TODO: add examples and link diagnostics tool.

missing boolean values ( not yet used but a good thing to have around I hear)